Skip to content

feat: Nvim can detect venv python via "pynvim-python" tool #594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 16, 2025

Conversation

drmikehenry
Copy link
Contributor

@drmikehenry drmikehenry commented Aug 9, 2025

Problem: Detection of the pynvim module is currently done by finding the first Python interpreter in the PATH and checking if it can import pynvim. This has several effects:

  • Activation of an unrelated Python virtual environment will break automatic detection, unless pynvim is also installed in that environment.

  • Installing pynvim to the expected location is difficult. User installation into the system-wide or user-wide Python site area is now deprecated. On Ubuntu 24.04 with Python 3.12, for example, the command pip install --user pynvim now fails with the error message error: externally-managed-environment.

  • Users may create a dedicated virtual environment in which to install pynvim, but Neovim won't detect it; instead, they must either activate it before launching Neovim (which interferes with the user of other virtual environments) or else hard-code the variable g:python3_host_prog in their init.vim to the path of the correct Python interpreter. Neither option is desirable.

Solution: Expose pynvim's Python interpreter on the PATH under the name pynvim-python. In the typical flow:

  • User installs either uv or pipx.

  • User installs pynvim via:

    uv tool install --upgrade pynvim
    # Or:
    pipx install --upgrade pynvim
    

With corresponding changes in Neovim (neovim/neovim#35273), the above is all that's needed for Neovim to detect the installed location of pynvim, even if an unrelated Python virtual environments is activated. It uses standard Python tooling to automate the necessary creation of a Python virtual environment for pyenv and the publication of pynvim-python to a directory on PATH.

I've aimed to update all relevant portions of the documentation. I'm happy to make adjustments, or you may feel free to make changes as needed.

Close #593

@justinmk
Copy link
Member

Are any of the CI failures related?

Problem: Detection of the pynvim module is currently done by finding the
first Python interpreter in the `PATH` and checking if it can import
pynvim.  This has several effects:

- Activation of an unrelated Python virtual environment will break
  automatic detection, unless pynvim is also installed in that
  environment.

- Installing pynvim to the expected location is difficult. User
  installation into the system-wide or user-wide Python site area is now
  deprecated.  On Ubuntu 24.04 with Python 3.12, for example, the
  command `pip install --user pynvim` now fails with the error message
  `error: externally-managed-environment`.

- Users may create a dedicated virtual environment in which to install
  pynvim, but Neovim won't detect it; instead, they must either activate
  it before launching Neovim (which interferes with the user of other
  virtual environments) or else hard-code the variable
  `g:python3_host_prog` in their `init.vim` to the path of the correct
  Python interpreter.  Neither option is desirable.

Solution: Expose pynvim's Python interpreter on the `PATH` under the
name `pynvim-python`.  In the typical flow:

- User installs either uv or pipx.

- User installs pynvim via:

      uv tool install --upgrade pynvim
      # Or:
      pipx install --upgrade pynvim

With corresponding changes in Neovim, the above is all that's needed for
Neovim to detect the installed location of pynvim, even if an unrelated
Python virtual environments is activated.  It uses standard Python
tooling to automate the necessary creation of a Python virtual
environment for pyenv and the publication of `pynvim-python` to a
directory on `PATH`.

See neovim#593 for additional
discussion of this idea.
@drmikehenry
Copy link
Contributor Author

I've added docstrings to pynvim/python.py, updated the commit message to avoid starting with a capital letter, and squashed these changes into the branch.

Tests and QA are passing on my local machine (invoked as uv tool run tox run -e py311,checkqa). This uses Neovim v0.11.3 (my installed version).

Testing against the pynvim-python branch of Neovim has one unrelated failure; the test also fails against Neovim master. I'm testing by activating the pynvim virtual environment and running:

VIMRUNTIME=/home/mike/projects/neovim/runtime PATH="/home/mike/projects/neovim/build/bin:$PATH" python3 -m pytest

The failing test is in test/test_window.py, function test_options(). The test is intended to demonstrate that a local value may be set for a global-local option without changing the global value. The test code is:

def test_options(vim: Nvim) -> None:
    vim.current.window.options['colorcolumn'] = '4,3'
    assert vim.current.window.options['colorcolumn'] == '4,3'
    # global-local option
    vim.current.window.options['statusline'] = 'window-status'
    assert vim.current.window.options['statusline'] == 'window-status'
    assert vim.options['statusline'] == ''

    with pytest.raises(KeyError) as excinfo:
        vim.current.window.options['doesnotexist']
    assert excinfo.value.args == ("Invalid option name: 'doesnotexist'",)

This chooses statusline as handy option to test, as it historically has had a known-empty default value and allows for setting an arbitrary test value.

I used git bisect to locate the change in Neovim that led to this failing test:

commit 28e31f5d3d16fac349d5e2b55837afddb822b0f3
Author: Shadman <[email protected]>
Date:   Tue Apr 22 04:05:34 2025 +0600

    feat(options): default statusline expression #33036

    Problem:
    Default 'statusline' is implemented in C and not representable as
    a statusline expression. This makes it hard for user configs/plugins to
    extend it.

    Solution:
    - Change the default 'statusline' slightly to a statusline expression.
    - Remove the C implementation.

The statusline option is no longer empty by default. The below patch (not part of this pull request) reads the default global option value before making a change to the local option, then verifies that the global option value is unchanged. With this patch, the test passes on Neovim nightly as well as stable.

diff --git a/test/test_window.py b/test/test_window.py
index 7a36d9e..9267a14 100644
--- a/test/test_window.py
+++ b/test/test_window.py
@@ -61,10 +61,11 @@ def test_vars(vim: Nvim) -> None:
 def test_options(vim: Nvim) -> None:
     vim.current.window.options['colorcolumn'] = '4,3'
     assert vim.current.window.options['colorcolumn'] == '4,3'
+    old_global_statusline = vim.options['statusline']
     # global-local option
     vim.current.window.options['statusline'] = 'window-status'
     assert vim.current.window.options['statusline'] == 'window-status'
-    assert vim.options['statusline'] == ''
+    assert vim.options['statusline'] == old_global_statusline

     with pytest.raises(KeyError) as excinfo:
         vim.current.window.options['doesnotexist']

@justinmk
Copy link
Member

The below patch (not part of this pull request) reads the default global option value before making a change to the local option, then verifies that the global option value is unchanged. With this patch, the test passes on Neovim nightly as well as stable.

Please add that as a separate commit here, so we can get a better idea of CI status.

The test previously depended on the 'statusline' option being empty by
default.  The default 'statusline' value changed in Nvim commit
28e31f5d3d16fac349d5e2b55837afddb822b0f3 (2025-04-22):

    feat(options): default statusline expression #33036

Adjust the test to first read the default (global) value of
'statusline', then change the local value of the option, then re-read
the global value and verify it hasn't changed.
@drmikehenry
Copy link
Contributor Author

I've added a commit to fix test_options().

@justinmk
Copy link
Member

The failing test_connect_stdio test was also failing on master, though the logs are no longer available.

@justinmk justinmk changed the title feat: Improve Neovim detection of pynvim. feat: Nvim can detect venv python via "pynvim-python" tool Aug 16, 2025
@justinmk justinmk merged commit b8899cb into neovim:master Aug 16, 2025
20 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Auto-detect pynvim located in dedicated virtual environment
2 participants